The present invention relates to electro-optical systems for pattern recognition, in general, and more particularly, to a system for correlating and integrating extracted image pattern areas from the image information of a plurality of frames to form a composite image pattern of at least one candidate object for classification in accordance with pre-specified image patterns of reference objects.
Typical image processing recognition systems involving image pre-processing, feature extraction and object classification are described in the following U.S. Patents:
U.S. Pat. No. 3,636,513, issued Jan. 18, 1972 to Glenn E. Tisdale and entitled "Pre-Processing Method and Apparatus for Pattern Recognition;" PA1 U.S. Pat. No. 3,638,188, issued Jan. 25, 1972 To Peter H. Pincoffs et al. and entitled "Classification Method and Apparatus for Pattern Recognition Systems;" and PA1 U.S. Pat. No. 3,748,644, issued July 24, 1973 to Glenn E. Tisdale and entitled "Automatic Registration of
Points in Two Separate Images," all being assigned to the same assignee as the instant application. The classification techniques of these type systems are generally based on the information contained in line segments which define a short contrast gradient boundary between contrasting regions in the image of a video frame. In these systems, classification accuracy is considered best when the candidate objects are relatively large and have high contrast to the adjoining background. However, under more adverse conditions, like when the object size is small due to long range, when the contrast is poor, when the signal-to-noise ratio (S/N) is low, or when partial obscuration or background merging with the candidate object occurs, for example, the classifier accuracy may be seriously degraded especially because under these adverse conditions, there may be a low number of valid line segments available and a general randomness of the line segments may exist as a result of noise.
Several techniques exist for reducing the sensitivity to random noise and many of them involve the use of integration methods. One such technique employs spatial filters which operate directly on the image of a video frame to replace the intensity amplitude of a specific picture element (pixel) of the image with another amplitude value derived as a function of the intensity amplitudes of the pixels in the neighborhood of the specific pixel to reduce the effect of random noise on the individual pixels of an image area constituting a portion of a video frame. While this filtering technique generally improves image quality of a video frame, it generally is limited to rather small pixel areas or neighborhoods of the video frame because image resolution quickly deteriorates as larger-area spatial filters are applied.
Another integration technique, generally referred to as temporal filtering, takes advantage of the relatively independent data provided by sequential video frames or samples of the same scene or field of view typically generated by electro-optical sensors included in many imaging systems. Some forms of integration utilizing temporal filter techniques operating on the image data of sequentially generated video frames at the pixel level are known to improve detection and classification performance. However, the process generally requires precision positional registration from frame-to-frame (i.e. a fraction of a pixel accuracy) for all of the video frames or samples to be integrated. Consequently, these techniques are considered very complex especially when both the electro-optical sensor and dynamic object motion within the scene are considered in the classification process. Nonetheless, just the combination of video frame data from all of the samples to be integrated at the pixel level requires significant computational capacity even without the aforementioned considerations.
Another filtering method sometimes used in pattern recognition and classification systems, generally referred to as output decision smoothing, involves combining the results of the classification process from several sequential frames to resolve possible conflicts therein. Usually, only corresponding objects are considered for correlation, thereby reducing greatly the registration requirements normally associated with integrating frame-to-frame information. However, the advantage provided by the decision smoothing process is very sensitive to the accuracy of the individual decisions of the classification process. Therefore, decision smoothing is generally considered useful when high individual classification accuracy can be achieved. Under realistic conditions, most existing classification techniques are not considered accurate enough to permit useful improvement by decision smoothing.
Apparently, what is needed is a simpler method of integrating image information from frame-to-frame which can eliminate the precision registration and reduce the complex processing generally required for the present integration methods. It is felt that the formation of an integrated composite image pattern of a candidate object will provide a more complete description thereof and minimize the effect of individual random pattern features caused by noise. It is expected that the composite image pattern will naturally reinforce the primary features of a candidate object, since they occur more frequently, thus improving the classification process. Furthermore, if there is obscuration or background merging with respect to a candidate object in a video frame, any sample-to-sample integration will tend to result in a more complete object definition in an integrated composite image pattern than would be available from an individual video frame. Still further, with an improvement in classification accuracy, especially under adverse conditions where performance is most critical, decision smoothing may become a useful and viable extension to the classification process providing further improvements thereof even under realistic conditions.